[Files] Add user ID to file by jloleysens · Pull Request #144026 · elastic/kibana

jloleysens · 2022-10-26T12:07:32Z

Screenshot

Adds the user flattened field to the base files metadata. This contains id and name.

Blocked by: #144272

Checklist

Unit or functional tests were updated or added to match the most common scenarios

elasticmachine · 2022-10-26T12:07:35Z

Pinging @elastic/kibana-app-services (Team:AppServicesUx)

Dosant · 2022-10-31T11:00:56Z

+    alt,
+    meta,
+    mime: mimeType,
+    owner: security?.authc.getCurrentUser(req)?.username,


I am not 100% sure that just username is enough to identify the user, as I saw this in the search code:

kibana/src/plugins/data/common/search/session/types.ts

Lines 53 to 58 in 7a3243b

/**

* The realm type/name & username uniquely identifies the user who created this search session

*/

realmType?: string;

realmName?: string;

username?: string;

I'd ask @elastic/kibana-security for a review if we haven't already

This is a good point @Dosant . We reached out to security team and they advised us to use profile_uid. Changes incoming.

jloleysens · 2022-10-31T11:18:30Z

+     * for searching and filtering.
     */
-    Meta?: Meta;
+    Meta?: Meta & FileSystemMetadata;


This was added to the Meta field because I was running into the 1000 fields upper limit for the SO index

Log link

Happy to use the regular declaration for new fields, but since we already have a flattened field that kind of fits this use case, I pivoted the implementation to use that instead. cc @vadimkibana

Also notified core about this limit being hit in our functional tests

A there any downsides of storing it under meta? Are these still searchable/aggregatable / filterable?

jloleysens · 2022-10-31T12:16:58Z

A there any downsides of storing it under meta? Are these still searchable/aggregatable / filterable?

Here is a list of the downsides: https://www.elastic.co/guide/en/elasticsearch/reference/master/flattened.html#supported-operations

The fields are essentially the same as "keyword" fields which for usernames and user IDs should work quite well.

vadimkibana · 2022-10-31T13:17:45Z

The problems of putting those fields in meta are:

Then those fields don't follow ECS anymore.
meta should be left for our customers to use to store their custom metadata, we should use something else to store our data.
- Those fields should be read-only, our consumers should not be able to edit user profile ID and username, by editing the meta field.

For profile ID, ideally we would use file.owner or file.user.id, which both are per ECS, AFAIK.

* main: Upgrade @elastic/makelogs from v6.0.0 to v6.1.1 (elastic#144231) [Files] move to src (elastic#144044) [Synthetics UI] Add pagination and date filtering to test runs table (elastic#144029) Update time range when opening timeline from Entity Analytics page (elastic#144024) [Security Solution] Added guided onboarding for the rules area (elastic#144016)

* main: (43 commits) [Synthetics] Step details page screenshot (elastic#143452) [Lens] Datatable expression types improvement. (elastic#144173) [packages/kbn-journeys] start apm after browser start and stop after browser is closed (elastic#144267) [Files] Make files namespace agnostic (elastic#144019) Implement base browser-side logging system (elastic#144107) Correct wrong multiplier for byte conversion (elastic#143751) [Monaco] Add JSON syntax support to the Monaco editor (elastic#143739) CCS Smoke Test for Remote Clusters and Index Management (elastic#142423) [api-docs] Daily api_docs build (elastic#144294) chore(NA): include progress on Bazel tasks (elastic#144275) [RAM] Allow users to see event logs from all spaces they have access to (elastic#140449) [APM] Show recommended minimum size when going below 5 minutes (elastic#144170) [typecheck] delete temporary target_types dirs in packages (elastic#144271) [Security Solution][Endpoint] adds new alert loading utility and un-skip FTR test for endpoint (elastic#144133) [performance/journeys] revert data_stress_test_lens.ts journey step (elastic#144261) [TIP] Use search strategies in Threat Intelligence (elastic#143267) Optimize react-query dependencies (elastic#144206) [babel/node] invalidate cache when synth pkg map is updated (elastic#144258) [APM] AWS lambda estimated cost (elastic#143986) [Maps] layer group wizard (elastic#144129) ...

Dosant

LGTM!

thomheymann

Nice one, looks great and is consistent with ECS / naming convention in audit log!

kibana-ci · 2022-11-07T10:09:49Z

💚 Build Succeeded

Metrics [docs]

Saved Objects .kibana field count

Every field in each saved object type adds overhead to Elasticsearch. Kibana needs to keep the total field count below Elasticsearch's default limit of 1000 fields. Only specify field mappings for the fields you wish to search on or query. See https://www.elastic.co/guide/en/kibana/master/saved-objects-service.html#_mappings

id	before	after	diff
`file`	10	11	+1

Unknown metric groups

API count

id	before	after	diff
`files`	276	278	+2

ESLint disabled in files

id	before	after	diff
`osquery`	1	2	+1

ESLint disabled line counts

id	before	after	diff
`enterpriseSearch`	19	21	+2
`fleet`	58	64	+6
`osquery`	108	113	+5
`securitySolution`	440	446	+6
total			+19

Total ESLint disabled count

id	before	after	diff
`enterpriseSearch`	20	22	+2
`fleet`	66	72	+6
`osquery`	109	115	+6
`securitySolution`	517	523	+6
total			+20

History

💚 Build #85392 succeeded ddeea1d
💚 Build #85076 succeeded f6c663e
💚 Build #84990 succeeded 24d9c72
💚 Build #84663 succeeded 6db3f3e
💔 Build #84372 failed 5556d7f
💔 Build #84283 failed 403b47d

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

jloleysens added 5 commits October 26, 2022 13:25

added the createdBy field to schemas

12b2110

update serialize-deserializer

f3bc75e

use the security plugin to get the user ID from the request object

755689c

get the user name from request

5ea2574

update tests

3462c39

jloleysens added release_note:skip Skip the PR/issue when compiling release notes auto-backport Deprecated - use backport:version if exact versions are needed Team:AppServicesUx Feature:Files v8.6.0 labels Oct 26, 2022

jloleysens requested a review from a team as a code owner October 26, 2022 12:07

jloleysens changed the title ~~[Files] Record created by field~~ [Files] Record created_by field Oct 26, 2022

jloleysens and others added 3 commits October 26, 2022 16:22

Merge branch 'main' into files-record-created-by

78559b9

Merge branch 'main' into files-record-created-by

f9075da

CreatedBy -> owner

2f3d2be

jloleysens changed the title ~~[Files] Record created_by field~~ [Files] Add user ID to file Oct 31, 2022

jloleysens added 2 commits October 31, 2022 11:39

make owner field optional

ce6954f

updated files SO hash

de4fcc6

jloleysens requested a review from a team as a code owner October 31, 2022 10:45

Dosant reviewed Oct 31, 2022

View reviewed changes

jloleysens added 2 commits October 31, 2022 12:15

record both username and UID

09b3061

actually record profile_uid!

fb4c3e2

jloleysens commented Oct 31, 2022

View reviewed changes

jloleysens added 2 commits October 31, 2022 12:20

only call getCurrentUser once

52d63a1

revert to previous files SO hash

41988d9

jloleysens removed the request for review from a team October 31, 2022 11:23

jloleysens enabled auto-merge (squash) October 31, 2022 11:29

jloleysens disabled auto-merge October 31, 2022 11:29

jloleysens enabled auto-merge (squash) October 31, 2022 11:30

fix expects for update arg and pass through meta argument

8f24d18

Dosant approved these changes Oct 31, 2022

View reviewed changes

jloleysens disabled auto-merge October 31, 2022 13:03

Dosant self-requested a review October 31, 2022 15:52

jloleysens and others added 13 commits November 1, 2022 11:07

added user field back to base metadata

2d7b3c1

remove references to FileSystemMetadata

46531e2

added user field as flattened

bafc2ec

updated files SO

9c1be50

update types and metadata

f63bb05

update how we record the user on the controller

b5ce91c

update tests

44ec96a

remove extra long comment from the Meta field

1a922aa

Merge branch 'main' into files-record-created-by

5556d7f

Merge branch 'main' into files-record-created-by

6db3f3e

Merge branch 'main' into files-record-created-by

24d9c72

Merge branch 'main' into files-record-created-by

f6c663e

Dosant approved these changes Nov 3, 2022

View reviewed changes

jloleysens enabled auto-merge (squash) November 4, 2022 10:16

Merge branch 'main' into files-record-created-by

ddeea1d

thomheymann approved these changes Nov 4, 2022

View reviewed changes

Merge branch 'main' into files-record-created-by

aa22298

pgayvallet approved these changes Nov 7, 2022

View reviewed changes

jloleysens merged commit 1428251 into elastic:main Nov 7, 2022

kibanamachine added the backport:skip This PR does not require backporting label Nov 7, 2022

jloleysens deleted the files-record-created-by branch November 7, 2022 10:46

	/**
	* The realm type/name & username uniquely identifies the user who created this search session
	*/
	realmType?: string;
	realmName?: string;
	username?: string;

Conversation

jloleysens commented Oct 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Screenshot

Checklist

Uh oh!

elasticmachine commented Oct 26, 2022

Uh oh!

Dosant Oct 31, 2022

Choose a reason for hiding this comment

Uh oh!

jloleysens Oct 31, 2022

Choose a reason for hiding this comment

Uh oh!

jloleysens Oct 31, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jloleysens Oct 31, 2022

Choose a reason for hiding this comment

Uh oh!

Dosant Oct 31, 2022

Choose a reason for hiding this comment

Uh oh!

jloleysens commented Oct 31, 2022

Uh oh!

vadimkibana commented Oct 31, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Dosant left a comment

Choose a reason for hiding this comment

Uh oh!

thomheymann left a comment

Choose a reason for hiding this comment

Uh oh!

kibana-ci commented Nov 7, 2022

💚 Build Succeeded

Metrics [docs]

Saved Objects .kibana field count

API count

ESLint disabled in files

ESLint disabled line counts

Total ESLint disabled count

History

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

jloleysens commented Oct 26, 2022 •

edited

Loading

jloleysens Oct 31, 2022 •

edited

Loading

vadimkibana commented Oct 31, 2022 •

edited

Loading